Many computers use file management software to access data stored in files stored on storage devices (e.g., hard disc drives (HDDs), solid state drives (SSDs), etc.). The file management software manages requests by applications to open and close files, create and delete files, read data from files, write data to files, etc. To perform these and other file management operations, the file management software may maintain file system records indicating one or more locations in a storage device where a file or parts thereof are stored. When an application requests access to a file or a particular portion thereof, the file management software uses these file system records to determine the address of the requested data on the storage device, transmits the appropriate signals to the storage device to obtain the requested data, and passes the requested data to the application. In various computer systems, the file management operations are typically performed by an operating system running on a general-purpose, programmable processing device (e.g., a central processing unit (“CPU”)).
Computer systems generally access data from one or more files during the execution of software applications such as applications for audio/video processing, financial transactions, data mining, etc. The CPU that performs the file management operations may additionally execute one or more software applications, or a computer system may include additional processors for executing the software applications. Some computer systems employ specialized processors such as application specific integrated circuits (ASICs), reconfigurable processing units (e.g., field programmable gate array (FPGA) processing units), etc., for executing the software applications or certain portions thereof. Such specialized processors can be faster and/or more efficient than general purpose processors and, as such, are often used to execute computationally intensive tasks and/or tasks that manipulate large amounts of data (e.g., tens or hundreds of megabytes, gigabytes, terabytes, or even larger sizes of data).
The ASICs and/or reconfigurable (e.g., FPGA-based) processing units generally rely on conventional file management software for accessing data in files stored on storage devices. Therefore, computer systems that include specialized processors typically include a general purpose CPU which executes an operating system with conventional file management software. Generally in such systems, the CPU uses the conventional file management software to obtain data from a storage device on behalf of a specialized processor and may store the data in a memory that is directly accessible to the specialized processor or may supply the data directly thereto. Likewise, after the specialized processor computes a result, the result may be stored in the memory directly accessible to the specialized processor for retrieval by the CPU or may be sent directly to the CPU. The CPU then uses the conventional file management software to store the result in a file stored on the storage device.
Generally in such computing systems, an operating system (OS) maintains file system data that indicates, for each of one or more files, where the files (or portions thereof) are stored and/or are to be stored on one or more storage devices (e.g., a hard disk, SSD, etc.). A file may be stored in a contiguous or non-contiguous set of storage units, often called blocks, in a single storage device or across several storage devices. The file system data typically includes for each file an identifier of the storage device where the file or a portion thereof is stored/to be stored, logical block(s) of file, and physical location(s) (i.e., address(es)) of corresponding physical unit(s)/block(s) of the storage device(s).
One example of a file system well suited for storing and accessing large files (e.g., files of sizes on the order of tens or hundreds of megabytes, gigabytes, terabytes, or more) is an extent-based file system. Extent-based file systems are well known and have widespread use in contemporary computing systems. In an extent-based file system, an extent includes one or more contiguous units/blocks of storage. The extent-based file system allocates one or more extents to a file, which tends to reduce file fragmentation and/or can speed up file access, because an entire large file or at least a significant portion thereof can be stored in a single extent. The allocation of extents often limits the amount of data in the file system directory, as well.
When a processor (e.g., a CPU) needs to access one or more data elements from a file and/or is ready to store such data element(s) in a file, the OS generally determines the logical file block(s) corresponding to such data element(s), computes the physical address(es) corresponding to those logical block(s) using the file system, and accesses and/or stores the data element(s) using the physical address(es). In a computing system that includes one or more specialized processor(s), such specialized processor(s) are typically not configured to generate and/or use file system information and/or to access data using such information. Instead, the specialized processors typically rely on a CPU/OS to perform file access operations, i.e., reading, updating, and/or writing one or more data element(s) from/to a specified file. This may create a performance bottleneck because data processing by the specialized processor(s) becomes dependent on the CPU/OS.
One solution is to configure one or more specialized processors to generate and/or use the file system information and/or to provide file access functionality. This, however, can increase the size, complexity, and/or cost of the specialized processor(s) and may also decrease their performance. Moreover, if only one or a few specialized processors(s) are configured to provide the file access functionality, the performance bottleneck may still exist because other specialized processor(s) that are not configured to provide the file access functionality may need to depend on the specialized processor(s) that are so configured and or on the CPU/OS. If many or all specialized processor(s) are configured to provide the file access functionality, in addition to likely increase in size, complexity, cost, and/or processing time, the specialized processor(s) may also become burdened with coordinating file access functionality among each other.